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Intervention 



The / CAN Learn® Mathematics Curriculum is an interactive software system for pre-algebra and algebra. It consists of 326 lessons 
from basic mathematics to advanced algebra concepts. Designed for students to learn at their own pace, the / CAN Learn® program 
is intended primarily for use in grades 7 through 10 by ethnically diverse, inner-city students. 

For Middle school students. 



Findings 



Evidence base 



Evidence limits 



Scope of use 



The one randomized controlled trial (254 students in one school) found that / CAN Learn® students scored significantly higher on the 
state math test than did students in the comparison group who used a “traditional” curriculum. Both quasi-experimental design 
studies found that / CAN Learn® students outscored comparison students. One study found that / CAN Learn® students scored 
higher than comparison students, but the differences were not statistically significant. The second study found that / CAN Learn® 
students scored higher than comparison students, but a flaw in the analysis makes it impossible to accurately determine the 
significance of the finding. 

^ 1 randomized controlled trial meets evidence standards, 
y 2 quasi-experimental design studies meet evidence standards with reservations. 

© 4 studies of this curriculum do not meet evidence screens. ( see S y m fo 0 / key on page 10) 

The evidence base for the / CAN Learn® Mathematics Curriculum includes three studies, only one of which was a randomized 
controlled trial. The others were quasi-experimental design studies, which provide weaker evidence of effects because 
unmeasured differences between the groups can affect the findings. None of the studies provided evidence of strong 
implementation. One quasi-experimental design study had flaws in analysis that preclude calculation of the confidence intervals 
around the estimated impacts of the intervention. 

The curriculum was first implemented in 1995. By October 2004 it was being used in 23 states by 501 schools. The software has 
typically been used in Title I schools and by at-risk and minority children in large urban school districts. 



Developer and contact JRL Enterprises, Inc., 400 Poydras Street, Suite 100, New Orleans, LA 70130; telephone: (504) 263-1380; www.icanlearn.com. 
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Profile The / CAN Learn® Mathematics Curriculum is designed to help 
ethnically diverse, inner-city students in grades 7 through 10 
achieve parity in mathematics and problem-solving skills. It uses 
an interactive software program to teach pre-algebra and 
algebra. The / CAN Learn® Algebra 1 course, which was used in 
all of the studies included in this intervention report, consists of 
177 algebra lessons. The developer describes the curriculum as 
meeting National Council of Teachers of Mathematics standards 
and configurable to meet state and local grade-level 
expectations. 

Lessons are grouped much like those in a textbook chapter. 
Students receive verbal and visual assistance in progressing 
through the lesson. After completing a lesson, students take a 
cumulative review of the concepts taught, and teachers can 
monitor student progress through real-time assessment. The 
software also handles taking attendance, assigning homework, 
and administering and grading tests. Tests are randomly 
generated from a test database, so no two students receive the 
same test. Similarly, homework is generated for students, and 
the software conducts random homework checks to make sure 
the assignments are being completed. 

Implementation of the / CAN Learn® Mathematics Curriculum 
involves the creation of an / CAN Learn® classroom with 
computers, custom desks, software installation, and typically 
three years of support from the developer. Desks are set up in a 
“conference room” arrangement, which the developer argues 
encourages cooperative learning and group work. 

Teaching 

Although much of the instruction is individualized for students in 
the / CAN Learn® Mathematics Curriculum, teachers are 
expected to help determine the content of the lesson and other 
aspects of the class. At the beginning of the year, the teacher 
determines the homework assignments, lesson organization, 
lesson presentation, manipulatives, assessments, and grade 
evaluations. The program follows the teacher’s lesson plan and 
provides constant feedback to both the student and the teacher. 

As part of the / CAN Learn® system, teachers receive an initial 
two days of training in how to use the system. Teachers also 



have “best practices” training and unlimited access to training 
and on-site technical and pedagogical support in the classroom 
for three years. The / CAN Learn® personnel call or visit / CAN 
Learn® teachers weekly. 

Typical lesson 

Each / CAN Learn® lesson follows a six-part format: pretest, 
review, lesson presentation, guided practice, post-lesson quiz, 
and cumulative review. The pretest covers material from the 
upcoming lesson. If students miss one question on the pretest, 
they continue into the current lesson. If students get all the 
questions right on the pretest, they may advance to the next 
lesson if the teacher enables the software’s advancement 
option. 

In the review section, students are presented with 
prerequisite material that is necessary for understanding the 
current lesson. During the lesson, a “cyber-teacher” presents 
concepts and problems that increase in complexity as the 
lesson progresses. Students work out the problems presented 
in a notebook and compare their answers with the ones 
presented by the program. If students complete the problems 
incorrectly, the “cyber-teacher” is available to work through the 
problems in detail — to help students figure out where they may 
have made an error. 

The post-lesson quiz contains a variety of problem styles that 
the developer indicates conform to national standardized tests. 
The classroom teacher determines completion time, the number 
of questions, and their respective difficulty. 

After students complete a group of lessons, they go to a 
cumulative review, which presents three problems from each 
lesson. If more than one is incorrect, the lesson presentations for 
those areas are reviewed. 

Scope of use 

By October 2004 the / CAN Learn® Mathematics Curriculum was 
being used in 23 states by 501 schools. It is typically used in 
large urban school districts and smaller rural school districts 
whose students are predominantly at risk and members of 
minority groups. 
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Profile 

(continued) 



Study findings 



Strength of the 
evidence base 
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Cost 

The cost of an / CAN Learn® classroom depends on its management software, computer hardware, network wiring, 

configuration and terms of support. A typical full installation — furniture, and three years of comprehensive onsite educational 

30 workstations in a classroom with all curriculum and classroom and technical support — costs $300,000, a one-time expense. 



Randomized controlled trial 

One randomized controlled trial with 254 students in one school 
(Kirby 2004) found that / CAN Learn® students scored significantly 
higher on the state math test than students in the comparison 
group, which was using a traditional math curriculum. 

Quasi-experimental design studies 

One quasi-experimental design study (Brooks 1999) found that 
7th- through lOth-grade students scored higher on a 



researcher-developed test than students in the comparison 
group; however, due to limitations in the way the analysis was 
conducted, it is not possible to determine whether the 
difference is due to the curriculum or to chance. A second 
quasi-experimental design study (Kerstyn 2001) found that 
8th-grade / CAN Learn® students outscored comparison 
students on the FCAT, but the differences were not statistically 
significant. 



The WWC collected more than 800 studies for the Middle School 
Math Curriculum review. Seven were on the / CAN Learn® 
Mathematics Curriculum. Of these, one met standards while two 
studies met standards with reservations. The other four studies 
did not meet standards. They were quasi-experimental design 
studies that did not account for pre-existing differences between 
groups with matching or equating. 

Studies were rated according to the strength of their causal 
evidence. Studies that placed students into the intervention and 
comparison groups randomly (randomized controlled trials) without 
notable design or implementation flaws are classified as meeting 
evidence standards (© a ). Other studies that use comparison 
groups (quasi-experimental designs) and randomized control trials 
with notable flaws are classified as meeting evidence standards 
with reservations ( ✓ a ). 

Studies are further rated for intervention fidelity, outcome 
measures, breadth of evidence, reporting on subgroups, analysis, 
and statistical reporting. That information is provided in study 
reports, but does not affect the overall rating. 



None of these studies provided evidence of strong implemen- 
tation. Implementation was not discussed in one of the studies. 
And there were variations in implementation in the other two 
studies. 

All three studies used valid tests that appear aligned to the 
curriculum, including state tests in two of the three studies. The 
third study demonstrated the reliability and validity of its locally 
developed test. 

Although each study looked at a narrow population and 
settings, collectively they looked at and reported on 8th- through 
lOth-grade students of diverse racial and economic backgrounds 
in urban and rural districts. Two provided enough information to 
compute effects. One other study had flaws in the analysis that 
may affect the findings. 

Tables A3-A5 describe the outcome studies on the / CAN 
Learn® Mathematics Curriculum that meet WWC standards and 
meet WWC standards with reservations. For more detailed 
descriptions of the studies, see the Detailed Study Reports or 
Brief Study Reports. 



a See symbol key on page 10. 
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Table 1 
Effects 



Study 


Measure 


© Kirby 
2004 


GCRCT a 

(N=254 students) 


/ Brooks 
1999 


Criterion Referenced 
Algebra Test 
(N=3,479 students) b 


✓ Kerstyn 
2001 


Algebra 1 

FCAT d (N=1 6 classes) 




Algebra 1 honors 
FCAT d (N=1 6 classes) 




Pre-algebra (MJ-3) 
FCAT d (N=64 classes) 




Pre-algebra (MJ-3 
advanced) 

FCAT d (N=20 classes) 




Pre-algebra (MJ-3) 
MJ-3 Cumulative 
Test (l\l=66 classes) 



Favors 

comparison group 
-2.00 -1.50 



- 1.00 



Comparison 
group mean 

-0.50 0 0.50 

0.41 



Favors 

I CAN Learn® group 
1.00 1.50 2.00 



0.1 3 C 



0.39 



0.06 



0.24 



0.44 



0.10 

-i- 



Approximate percentile ranking 



2 % 



7% 



16 % 



31% 50% 69% 84% 93% 98% 



a Georgia Criterion Referenced Competency Test, 
b Sample size reported is unit of analysis, not unit of assignment. 

c When there is no solid line, the study did not provide data to correctly compute the confidence interval, 
d Florida Comprehensive Assessment Test. 

How to read this table: The wide, shaded bar indicates both the direction and estimated size of the effect of the intervention. The estimated effects reported here are standardized differences in the mean values 
between the intervention and comparison groups. Bars extending to the right of zero denote estimated effects that favor the intervention group and those extending to the left of zero denote estimated effects that favor 
the comparison group. The solid line through the shaded bar marks the 95% confidence interval of the estimated effect. When the line does not cross zero, the estimate is statistically significant. The bar is hatched if the 
effect is not significant or if significance could not be accurately computed. The scale at the bottom of the chart indicates the approximate percentile distribution of students in the control group. The percentile ranking at 
the end of the shaded bar can be used to interpret the standardized mean difference in the outcome. For example, an effect of .5 is roughly equivalent to an increase in the mean value from that of the average student in 
the comparison group (50th percentile) to that of the average student at the 69th percentile. 
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Appendix 



Table ai Summary characteristics and findings from randomized controlled trials on the / CAN Learn® Mathematics Curriculum 











Sample size 




Mean outcome 


Standard deviation 3 


Estimated impact b 




Study 




Intervention Comparison 




Intervention 


Comparison 


Intervention 


Comparison 


Mean 


Standardized 


Study 


sample 


Measure 


group 


group 


Total 


group 


group 


group 


group 


difference 


mean difference 


Kirby 


8th-graders in 


GCRCT C 


91 


163 


254 


333.5 


319.9 


35.7 


31.7 


13.6 


0.41 (±0.26) 


2004 


regular education 








students 















a Shows how dispersed the participants' outcomes are. A small standard deviation would suggest that participants had similar outcomes, 
b The WWC estimated impact based on statistics reported by the study author, 
c Georgia Criterion Referenced Competency Test. 
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Table A2 Summary characteristics and findings from quasi-experimental design studies on the / CAN Learn® Mathematics Curriculum 











Sample size 




Mean outcome 


Standard deviation 3 


Estimated impact" 


Study 


Study 

sample 


Measure 


Intervention Comparison 
group group 


Total 


Intervention 

group 


Comparison 

group 


Intervention 

group 


Comparison 

group 


Mean 

difference 


Standardized 
mean difference 


Brooks 

1999 


7th- through 
1 0th-graders 


A 30-item 
criterion 
reference 
algebra test 0 


2,230 


1,249 


3,479 

students' 1 


7.5 


6.9 


4.7 


4.0 


0.6 


0.1 3 e 


Kerstyn 

2001 


Algebra 1 


FCAT f 


8 


8 


16 

classes 


351.1 


345.4 


15.6 


11.4 


5.7 


0.39 (±0.99) 


Algebra 1 honors 




8 


8 


16 

classes 


374.2 


373.1 


11.1 


20.5 


1.1 


0.06 (±0.98) 




MJ-3 




32 9 


32 


64 

classes 


298.0 


294.4 


15.6 


13.8 


3.6 


0.24 (±0.49) 




MJ-3 advanced 




10 


10 


20 

classes 


331.5 


326.1 


12.6 


11.0 


5.4 


0.44 (±0.89) 




MJ-3 


MJ-3 

Cumulative Test 


33 


33 


66 

classes 


31.4 


30.9 


4.6 


5.1 


0.5 


0.10 (±0.48) 



a Shows how dispersed the participants’ outcomes are. A small standard deviation would suggest that participants had similar outcomes, 
b The WWC estimated impact based on statistics reported by the study author, 
c Test developed by evaluation team based on textbooks, 
d The sample size reported is unit of analysis, not unit of assigment. 

e The unit of analysis (students) did not match the unit of assignment. For that reason, accurate confidence intervals could not be computed, 
f Florida Comprehensive Assessment Test. 

g One student left the study after the MJ-3 Cumulative Test was administered and before the FCAT was administered. 
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Table A3 Characteristics of interventions in reviewed studies on the / CAN Learn® Mathematics Curriculum: Kirby 2004 



Evidence 

base 

rating 3 


Characteristic 


Description 


0 


Study citation 


Kirby, P. C. (2004). Comparison ofl CAN Learn® and traditionally-taught 8th grade student performance on the 
Georgia Criterion-Referenced Competency Test. Unpublished manuscript. 




Participants 


254 8th-grade students in Gilmer Middle School. The study was limited to regular education students. 




Setting 


One middle school in the Gilmer County school district in northwestern Georgia. The county has a population of 
25,000 and is located in the Northwest Georgia mountains. 




Intervention 


Students were taught using the / CAN Learn® Mathematics Curriculum. The author does not indicate how many of 
the lessons are required to be completed for the curriculum to be implemented as intended. In this study the 
intervention lasted for one school year. 




Comparison 


Classes were taught using a traditional math curriculum. The author does not provide further information on the 
curriculum. 




Primary oucomes and 
measurement 


Georgia Criterion Reference Competency Test (GCRCT), a state test. The GCRCT math test was implemented in 
Georgia in 2000 in grades 4, 6, and 8 and in 2002, in grades 1,2,3, 5, and 7. The math test contains 60 items in six 
areas: number sense and numeration, geometry and measurement, patterns and relationships/algebra, statistics and 
probability, computation and estimation, and problem-solving. 




Teacher training 


There was only one / CAN Learn® teacher in this study. She was trained the prior year. 



a See symbol key on page 10. 
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Table A4 Characteristics of interventions in reviewed studies on the / CAN Learn® Mathematics Curriculum: Kerstyn 2001 



Evidence 

base 

rating 3 


Characteristic 


Description 


• 


Study citation 


Kerstyn, C. (2001). Evaluation of the 1 CAN Learn® mathematics classroom: First year of implementation (2000-2001 
school year). (Available from the Division of Instruction, Hillsborough County Public Schools, Tampa, FL) 




Participants 


8th-grade students in Title 1 middle schools. The study was limited to regular education students. The students were 
racially diverse and many were eligible for free or reduced-price lunches. There were 58 intervention classrooms and 
58 comparison classes used in the analysis. 




Setting 


36 middle schools in the Hillsborough County Public School system in Florida. This county includes the Tampa metro 
area. 




Intervention 


Students were taught using the / CAN Learn® Mathematics Curriculum. The author does not indicate how many of 
the lessons are required to be completed for the curriculum to be implemented as intended. The author indicates that 
the / CAN Learn® Mathematics Curriculum was implemented in class periods of 45, 50, 80, and 90 minutes in 
length. When surveyed, the teachers reported that 45 minutes was not long enough to make it through the 
curriculum. In this study, the intervention lasted for one school year. 




Comparison 


Classes were taught using a traditional math curriculum. The author does not provide further information on the 
curriculum. 




Primary outcome and 
measurement 


Florida Comprehensive Assessment Test, a state test, and MJ-3 Cumulative Test (an end of semester exam 
administered at the end of the first semester). The MJ-3 Cumulative Test is not a state or nationally normed, 
standardized test, but it has established reliability. 




Teacher training 


Teachers in this study participated in training sessions on the use of the software and hardware, but not on use of 
the software in instruction. 



a See symbol key on page 10. 
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Table as Characteristics of interventions in reviewed studies on the / CAN Learn® Mathematics Curriculum: Brooks 1999 



Evidence 

base 

rating 


Characteristic 


Description 


• 


Study citation 


Brooks, C. (1 999, August). Evaluation of Jefferson Parish Technology grant: 1 CAN Learn® Algebra 1. (Available from 
the Department of Educational Leadership, University of New Orleans, New Orleans, LA 70148) 




Participants 


3,479 students enrolled in 7th- through 1 0th-grades and came from honors, gifted, and remedial classes. There 
were 1 02 intervention classes and 67 comparison classes at the start of the study. 




Setting 


21 schools (20 public, 1 parochial) with five school districts in Louisiana. 




Intervention 


Students were taught using the / CAN Learn® Mathematics Curriculum. The author does not indicate how many of 
the lessons are required to be completed for the curriculum to be implemented as intended. In this study the 
intervention lasted for one school year. 




Comparison 


Classes were traditionally taught. No other information provided. 




Primary outcomes and 
measurement 


30-item criterion referenced achievement test developed by the evaluation team based on student textbooks. This 
test is not a state or nationally normed, standardized test, but it has established reliability. 




Teacher training 


Teachers received training before the start of the 1 998/99 school year. 



Symbol key 
for evidence 
base rating 



© Study meets evidence standards (randomized controlled trial without notable flaws). 

7 Study meets evidence standards with reservations (randomized controlled trial with notable flaws or quasi-experimental 
design study without notable flaws). 

Q Study does not meet evidence screens. 
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